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[57] "ABSTRACT 

A method and system is provided for the analysis of a 
human subject's brain wave activity on a statistical basis 
using a digital computer. The location of portions of the 
subject's brain and the location of at least 32, and for 
example 265, external scalp sensors are digitally re- 
corded and stored in computer memory. The subject 
receives a set of stimuli which evoke brain waves 
(evoked potential EP or evoked magnetic fields EF) 
which, along with the location data, are statistically 
analyzed to indicate the brain sites giving rise to the 
activity. The brain site activity, and the time interrela- 
tionships of brain waves as they progress between brain 
sites, are displayed on a three-dimensional model or 
other three-dimensional perspective display. 
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- _ waves into digital data. The three-dimensional positions 

BRAIN WAVE SOURCE NETWORK LOCATION of the electrodes are measured with a digitizer which 
SCANNING METHOD AND SYSTEM records their positions and stores their positions in com- 

. . puter storage. A magnetic resonance image (MRI) of 

This invention was made with Government support 5 each subject is also recorded and the electrode sensor 
under AFOSR Contract F49620-84-K-0008 awarded by positions are mathematically registered, in computer 
the United States Air Force. The Government has cer- storage, with the scalp in the MRI image. A computer- 
tain rights in this invention. controlled set of adaptive tasks are presented to the 
BACKGROUND OF THE INVENTION subject in order to extract functional source network 

10 patterns. These tasks preferably include: a bimanual 

A number of scientific articles have been published (two-hand) visuomotor task (hand response to a picture) 
regarding experiments directed to associating specific with and without load on short-term memory; an audi- 
activity, such as vision, with specific areas of the human tory- visual visuomotor task, such as the subject's mov- 
brain. Some of these experiments involve invasive tech- ing his finger when he hears a specific word, or sees the 
niques on animals, for example, the use of implanted 15 correct picture on the TV monitor; an auditory moni- 
electrodes or chemical dyes. toring task with varying load on short-term memory; 

However, other studies have taken a different ap- temporal sequencing task; and a linguistic battery. The 
proach and have attempted to discover what happens in electroencephalographic (EEG), and, optionally mag- 
a person's brain when a person performs a task, like netoencephalographic (MEG), brain waves and eye 
raising his finger, by placing electrodes or other sensors 20 movement (EOG) data are scanned for noise contami- 
on or near the person's scalp. Those electrodes or sen- nants, such as muscle artifacts, by several different lay- 
sors detect the tiny, microvolt or femtotesla level, elec- ered-network statistical pattern classifiers. Then, on- 
trical or magnetic activity at the surface of the scalp. line, a technician uses interactive computer graphics to 

The detection of such faint brain wave signals is diffi- check the borderline trials and eliminate those with top 
cult because of the noisy electrical and magnetic envi- 25 much noise. Using a statistical pattern classification 
ronment created by hundreds of cars, electric lights, algorithm, the signal-to-noise ratio can be further im- 
etc., etc. In addition, physiological "artifacts", which proved by eliminating individual trials which lack 
are electrical signals arising, for example, from scalp strong event related signals. Bandpass digital filters are 
muscle activity or the eye blink of the subject, may then applied to further enhance event related signals of 
produce an electrical and magnetic wave which is 30 interest. 

stronger than the subject's brain waves. The data are now free of contaminants and have a 

The detection and analysis of brain waves is even satisfactory signal-to-noise ratio in each trial. However, 
more difficult when the interest is in locating the spe- the EEG data are still spatially blurred (out of focus), 
cific three-dimensional sites giving rise to the waves and and must be further processed prior to spatial analysis, 
how the sites communicate with each other deep within 35 Consequently, spatial deconvolution analysis is applied 
the brain, as the subject responds to stimuli to the EEG data to reduce the distortion of the brain 

As an example of stimulus presentation, a subject is waves due to their transmission through the cerebrospi- 
placed before a TV monitor. The letter "V" appears. If nal fluid (internal brain fluid), skull and scalp. The spa- 
rt slants to the left, the subject is to get ready to push a tial deconvolution analysis is based on a finite-differ- 
button with his left hand when a number appears on the 40 ence model of the brain and head. When there are fewer 
monitor, and, if it slants to the right, he should get ready than 64 electrodes and the distance between electrode 
to push a button with his right hand. This is a simple sensors is greater than 3 cm, an optimal estimate of the 
task. But it has proven difficult to localize and display, Laplacian operator in two dimensions can be substi- 
m a readily comprehendable form, the dynamic brain tuted for the deconvolution. 

wave activity ("mass neuroelectric processes") which 45 "Functional relationships" between sensors are next 
give rise to the subject's ability to recognize the stimuli, computed on averages obtained from the enhanced, 
lc, to recognize the "V" and the subsequent number filtered, deblurred EEG or MEG data. The functional 
(cognitive behavior) and to physically react (percep- relationships are computed as the lagged crosscovari- 
tumotor), i.e., move his hand. ance over fraction-of-a-second intervals time registered 

There are not now presently commercially available 50 to a stimulus or to a response. The EEG or MEG cur- 
any non-invasive methods or systems which will locate rent distributions and the crosscovariance information 
selected brain sites, and display what occurs at those are then displayed in a three-dimensional appearing 
sites m response to stimuli and responses, in an accurate perspective display, preferably in color, which shows 
three-dimensional appearing display. stimulus and response-related "functional relationships" 

SUMMARY OF THE INVENTION 55 between s^sors at each of a number of time intervals. 

The data, at this step in the method, have not yet been 

In accordance with the present invention, there is processed to determine whether the brain waves reflect 
provided a method and system to detect, store, analyze near surface brain activity from the immediately under- 
and display a subject's brain waves, called "MAN lying areas, from distant near surface areas, or are de- 
SCAN", for a mental activity network scanner, FIG. 4. 60 rived from sites deep within the brain. Therefore a 

A large number of electrode sensors, at least 32 and as source network localization analysis is performed. This 
many as 256, are placed proximal to the scalp of the involves using linear-least-squares analysis to determine 
subject, preferably using a hat In other systems, for the optimal match between the brain wave fields ob- 
example, conventional electroencephalographic (EEG) served at the scalp and the fields produced by different 
recording, it is usual to use only about 20 electrodes. 65 combinations of model sources. Either EEG or MEG 
Each electrode is connected, through a low-noise high- field measurements, alone or in combination, can be 
gam amplifier, to an analog-to^digital multiplexer sys- used in this analysis. A finite-difference model of the 
tem which converts the subject's faint analog brain subject's brain and head is constructed from magnetic 
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positions of all the electrodes of the helmet 10, for ex- points can be registered with a simple translation and 

ample, even with 256 electrodes ten minutes should orthogonal rotation: 

suffice. From this information as to the three-dimen- This procedure produces estimates of the scalp sur- 

sional electrode positions and the known length of elec- face points derived from the EEG and/or MEG elec- 
trodes, the scalp location of each recording electrode 5 trode sensor positions and the MRL Numerical compar- 

tip is calculated by the computer, for example, a Mas- ison of these estimates then provides the transformation 

scomp 550 computer. (scaling, rotation and translation) required to take one 

- n . . .. . 10 set of digitized coordinates into the other. This allows a 

C. Positions of Cranial and Cortical Structures double-check on the accuracy of positioning using the 

A full set of clinical magnetic resonance image (MRI) 10 fiducial markers, 

images is obtained for each subject before the helmet is The magnetoencephalograph (MEG) is an alternative 

placed on the subject. These are high-resolution (0.95 system to the EEG to detect brain waves, 

mm pixel), 3-axis (coronal, sagittal and transverse) im- The weak magnetic fields from the human brain may 

ages of the head of each subject which show a cross-sec- °e detected and mapped using a cryogenic "SQUID" 
tion of the proton density at sections as little as 3 mm 13 (superconducting quantum interference device). The 

apart over the whole volume of the head. The MRI brain activity evoked by sensory stimuli produces weak 

tapes are read into the computer and displayed on a niagnetic fields in the order of 0.1 pT (picotesla). The 

color graphics terminal. Certain head areas of the stored resolution is limited by the size of the pick-up coil; 

MRI images ('loci") are digitized to give coordinate however, significant variations- in the magnetic field of 
surfaces for the scalp, the outer and inner surfaces of the 20 the brain have been observed at distances of 1 cm, see 

skull, and cerebral surface, as well as the loci of the Williamson, Brenner and Kaufman, Biomedical Applica- _ 

major fissures (e.g., Sylvania, central, longitudinal, cal- P 0 ** ofSQUJDSs (ISSN: 0094-243X/78/106/1.50 Am. 

carine), see FIG. 5. Inst of Bwca, pages 106-1 16, 1978), incorporated by 

All MRI images contain an inherent non-linear dis- reference herein. The SQUID senses the magnetic field 
tortion which can arise from several sources and may 25 ^ ^superconducting flux transporter whose detect- 

vary in magnitude and proportion depending on a num- mg ? oi ^ *^™e of its wmdmgs and geometry, is rela- 

ber of parameters. MRI distortion correction can be ^ely "^sensitive to uniform background fields gener- 

divided into two parts, (1) distortion measurement and ated by umfonn sources * 

characterization and (2) image correction. Before cor- 3Q E. Computer-Controlled Adaptive Tasks To Test 

rection, this distortion is measured using phantoms with Higher Brain Functions 

known physical dimensions. The position of the elec- a<w - + m , , ■ . 

trode sensors is marked by placingVmall oU (vitamin A . ^ thc helmet 10 is placed on the head of the sub- 

or E) capsules on an identical helmet which is then J ^V^ 

placed on^hTscalp. The capsule locations are clearly „ move some^portion of ms body m 

s^mtheMRIiiSageasXactem^^ whlch he ^es, such as a 

^XS^^^T ST ap ° lld t0 ^ff* 01 * l - Bimanual Task - ™»" measures the subject's 

of the electrodes with which they are associated. preparatory processes, visual stimulus processing, re- 

* ^f JOr ™~ °f ^ ortlo » arises from the ^ rae ^ on ^ updating to abo *' ^ 

n^mogeneity of the magnetic gradient coils of the 40 accuracy of his performance. In this task the subject is 

™ JTE? Sy8 ^ m * ^ oweve /' ^non-uniform static instructed to fixate on a point at the center of an amber 

nh^^ "T* v* ^Pf 1 ^^^ CRT monitor of a graphics processor, monitor type 

observed object positions. A characterization of the USI EV -2131A, which is positioned 70 cm directly in 

image deformation due to changes m the repetition time front 0 f his eye. He waits for a visually presented re- 

(TR) and echo time (TE) parameters is made m addition 45 spouse hand cue, for example, the letter V, slanting to 

to analysis of object dependent and object independent the left for a left-hand response and slanting to the right 

error. Time stability of the imaging system is typically for a right-hand response. The duration of the picture is 

tested by comparing reference MRI with those made 125 msec. The characters on the monitor screen have an 

during the lasat system recahbration. mumination of 0.5 log fL. One second later the stimulus 

Image correction requires fitting a spline to the em- 50 is presented on the CRT monitor having a duration of 

pincaUy measured or predicted distortion function. For 125 msec. The characters subtend a visual angle of 

each image the actual and observed object positions under one degree, also slanting to the left for a left-hand 

define a two-dimensional grid of points which charac- response or to the right for a right-hand response. A 

tenzes the distortion. A smooth surface spline which random 22% of the trials are miscued, Le., cued for a 

mterpolates each of the grid points is then constructed. 55 left-hand response and with stimulus for a right-hand 

FinaUy the distortion is corrected by mapping the image response, or vice versa. The subject is told to withhold 

through the inverse of this function using any of several response, not move his hands, on the miscued pictures, 

well-known image transformation algorithms. If he moves his hands on a miscued picture, the feed- 

D. Registration of Coordinates for MRI, EEG and/or M b& ? s *! ows " £ b f or that trial. 

MEG 60 In this set of trials, the subject is instructed to respond 
. to the stimulus, i.e., the picture on the monitor screen, 
After correcting the distortion, the MRIs are regis- without hesitation with a ballistic flexion of his right- or 
tered with the EEG and/or MEG data. The positions of left-hand index finger on a modified Grass Force Trans- 
known external fiducial marks (inion, nasion and exter- ducer, exerting a pressure corresponding to the stimulus 
nal pre-auricular notch) are determined in the MRI, 65 number of a linear scale of 1 to 9. For example, if the 
EEG and/or MEG coordinate systems (FIG. 6). When picture is left 3, he should respond with medium pres- 
the position of at least three identical points is known in sure with his left index finger. A two-digit feedback 
each of the coordinate systems, the remaining data number, of 125 msec duration, indicating the exact force 
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applied (to one-tenth of a unit) is presented on the moni- from 4 to 5 seconds. Stimuli are presented over two 

tor screen to the subject 1 sec. after his peak response speakers, which are located in front of the subject As in 

pressure. Trials are randomly ordered in blocks of 20, the VMMT tasks, the subjects respond by pressing on a 

except that the first two trials are not miscued. The force transducer with the forefinger of their right hand, 

subject initiates a block of 20 trials by pressing a button 5 In the AUM task, unlike the VMMT, the response is 

and the 20 trials follow automatically under computer ' registered as either on or off, and there is no feedback or 

program control, at inter-trial intervals of 1.5 seconds. warning signal for individual trials. Subjects receive 

If the subject does not respond within 1 sec, after the visual feedback, by viewing the CRT monitor screen, at 

stimulus, his response is ignored and the feedback is the end of each block of trials, showing: (1) targets 

presented to the subject on the monitor screen 2 sec. 10 (catch-trials) hit; (2) targets missed; (3) false positives; 

after the stimulus, indicating his non-response. After (4) average reaction time; and (5) standard deviation of 

each block of trials, a report appears on the CRT moni- reaction time. 

tor showing the average response accuracy for right 3. Sequencing Task. This task is designed to test the 

and left hands separately, the number of incorrect re- ability of the subject to temporally integrate and manip- 

sponses to miscues and the number of "bonus points". 15 ulate sequences of stimuli and produce sequences of 

2. Visuo-Motor and Auditory Motor Memory Tasks, motor responses. In this task, the subject sits facing a 

These tasks, called "working memory load tasks", test graphics terminal and waits for the appearance of a 

the subject's immediate working memory and percept- small **X" at center screen. The "X" signals the start of 

uometer coordination. Preferably there are three differ- a trial and serves as a visual fixation point. The stimulus 

ent memory load tasks presented to each subject, 20 sequence begins 1 second later. The sequence consists of 

namely: (1) the Visuo-Motor Monitoring Task, null 3-8 dots of light, 125 msec duration, presented at the 

memory condition (VMMT, 0-back); (2) the Visuo- rate of 4 per second. Each dot subtends a visual angle of 

Motor Monitoring Task, memory load condition 0.25 degree, and can appear in one of three positions: at 

(VMMT, 2-back); and (3) an Auditory Monitoring the center, 0.5 visual degree to the left, or 0.5 visual 

Memory (AUM) Task. 23 degree to the right. During a pretesting training period 

The VMMT is designed to allow precise control of the subject learns to associate right-hand index, middle 

stimulus parameters, load on immediate working mem- and ring Fingers with left, middle and right stimulus 

ory, motor activity and eye movements. Stimuli consist positions, respectively. Two hundred fifty msec after 

of single-digit numbers (125 msec duration) presented the last stimulus, an imperative symbol, a triangle of 

on the CRT monitor screen as in the Bimanual Task. 30 duration 125 msec, is presented at center screen. 

Subjects respond on each trial with a ballistic flexion of The task is to execute a sequence of finger presses on 

their right index finger on a Grass Force Transducer three switches under index, middle and ring fingers of 

with a pressure proportional to the stimulus number on the right hand, evenly timed and as close to the stimulus 

a linear scale from 1 to 9. The feedback number, shown presentation rate as possible (4/sec). In the "forward" 

after each trial to the subject on the monitor screen, is 35 condition (upward triangle) the subject is to execute a 

underlined if the response is sufficiently accurate. Accu- sequence of finger movements corresponding to the 

racy is based on an adaptive error tolerance, the geo- stimulus series. In the "backward" condition, inverted 

metric average of the error (the distance from the re- triangle, he is to execute the sequence in reversed (ret- 

quired response pressure) on the previous five move rograde) order. The fixation point remains on the screen 

trials. The inter-trial interval is 1.5 sec,, during which a 40 until the feedback interval in order to help the subject 

fixation symbol (X) is on the screen. fixate his eyes. The feedback display, presented 500 

The two memory load conditions in the VMMT are: msec after completion of the last response to the sub- 

a null memory or 0-back condition, in which the re- ject, shows the expected sequence on top with the se- 

quired response pressure is indicated by the current quence actually produced by the subject just under- 

stimulus number, and a 2-back memory condition, in 45 neath. At the end of every block of 15 trials, a report is 

which the response pressure is indicated by the stimulus shown with the percent of correct reponses, and a per- 

number which was given two trials back. In other formance goal which must be exceeded in the next 

words, the subject is told not to respond to the current block to win a bonus. In order to maintain a constant 

number or the number before, shown on the screen, but level of challenge, the number of dots in the sequence 

rather to the number which appeared two trials back. In 50 automatically increases as a moving average function of 

order to add a recognition element to the tasks and also the subject's performance during the preceding five 

to avoid overly short response times in the memory blocks. 

task, 20% of the trials in both conditions are designated 4. Language Tasks. A series of three language tasks 
as catch-trials in which the subjects are not required to measures the brief, time-varying neurocognitive pat- 
respond during these trials. In the 0-back condition 55 terns of the subject's basic phonemic, lexical and gram- 
(response to current number) the no-move catch-trials matical processes. The tasks involve: (1) matching pho- 
are those in which the stimulus number is 0. In the nemic stimuli divorced of sematic content (e.g.,/sen- 
2-back condition, catch-trials are designated as the oc- /and/cen/) versus stimuli comprised of letters which do 
currence of a stimulus number identical to the 2-back not form a phonemic sound (e.g., xkj); (2) identifying 
stimulus number. 60 homonyms (e.g., more and moor) versus antonymic 
The AUM (Auditory Monitoring Memory) task con- contrasts (e.g., old and new); and (3) assessing the gram- 
sists of a 2-back (or 3-back) memory load condition. In maticality of simple sentences on the basis of number, 
this task, the subjects are asked to respond whenever subject-verb and tense agreement, 
the current stimulus is the same as the number two (or During these language tasks, the stimuli are presented 
three) trials back. Stimuli consist of single-digit numbers 65 on a CRT monitor screen, or over loudspeakers, as in 
from 245 to 430 msec, spoken by a voice synthesizer, the other tasks. The subject responds by pressing 
such as the Votrax (TM) synthesizer, and presented in contact switches with his right hand, index and middle 
blocks of 50. The interstimulus interval varies randomly * fingers. A warning cue precedes the first stimulus by 1 
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sec, and the second stimulus is presented 1 sec. after the nal fluid, skull and scalp, and the passive transmission of 

first stimulus. The inter-trial interval is 2 seconds. Trials currents into the skull and then to the surface of the 

are presented in blocks of 20 randomly ordered task scalp is modeled, FIG. 7A. This operation is performed 

types. There is an equal number of match and mismatch without imposing an arbitrary source generator model; 

trials. After each block of trials, a performance score 5 the numbers), positions), or orientation(s) of possible 

report is provided, as described above. sources are not identified. The finite-difference ap- 

p Arrifiir* D»;~*Un proach to modeling the head hasa the advantage that 

r . Arniact Kejection ^ discretization can ^ performed with a grid of points 

Since the brain waves are so faint, at the microvolt or which can be uniform for each coordinate direction, 

femtotesla level, it is important that the recorded data 10 This procedure admits a simple interpretation in which 

be as free as possible from noise contaminants called the brain is viewed as an electrical circuit. The circuit 

"artifacts". consists of a network of equivalent resistors whose 

In order to reject various types of artifacts, on-line connecting nodes are the "uniform sampling points" of 

programs check for dead or saturated channels, monitor the model. 

the subject's eye movements, and delay the start of each 15 In the finite-difference implementation, the normal 
trial until the systems's amplifiers have recovered from currents entering the skull are calculated, by the corn- 
artifacts. When a run of 100 trials is completed, it is puter, from the observed brain wave potentials at the 
subjected to an automated artifact analysis using the electrode sensors using piece wise linear approximations 
computer under control of a computer program, Le., to the electrical properties of the skull and scalp re- 
"software". The artifact detection software detects and 20 gions. A finite difference form of the Laplace equation 
labels artifacted segments of single-trial time series. In for a sourceless volume yields Kirchoffs Current Law 
particular, programs check for vertical and horizontal at a network node. Using a 3-D reconstructions of each 
eye movement, eye blinks, dead or weak channels, satu- subject's head from MRI sections as stored in computer 
rated channels, and muscle potential contamination or memory, the equivalent resistive network for the skull 
other high-frequency noise that has significant energy 23 and scalp is derived by the computer software. Given 
in spectral regions of interest, see FIG. 7. These soft- certain restrictions on its topology, a sourceless resistive 
ware programs use a combination of time domain and network with known voltage outputs can be uniquely 
frequency domain features for classification by multiple solved for the entering currents. Those entering cur- 
l-layer pattern recognition machines. Parallel detectors rents are the brain waves produced at each of the se- 
are each specialized for a different type of artifact. Sim- 30 lected brain sites. 

pie amplitude detectors are sufficient ot detect dead Solution of the system node equations takes several 
channels and amplifier saturation. A layered-network seconds on a Masscomp 550 computer for 10,000 equa- 
pattern classifier uses low-frequency power spectral tions (nodes) and a sparse matrix frontwidth of 12, using 
density, cross spectral density with horizontal and verti- sparse Gaussian elimination. This yields a transforma- 
cal eye movement channels, and a correlation coeffici- 35 tion matrix which is stored in computer memory and 
ent with the vertical eye movement channel as candi- which is then applied to each distribution of measured 
date features for eye blinks and movement Another potentials, to derive the strength of the currents enter- 
classifier uses power spectral density, zero-lag auto-cor- ing and exiting the subject's scalp, 
relation, and the first-order autoregressive (AR) predic- 2. Conductivity Measurements in the Model A three- 
tor coefficient as candidates for muscles and instrumen- 40 dimensional appearing model, which may be viewed in 
tal artifact An optional third classifier uses the same perspective view, is used as the analog or constructed 
features for other types of artifact that users wish to twin of the subject's brain and head. The ratios of con- 
train the machine to reject Features are evaluated over ductivities in the model is determined by measuring the 
half-second intervals every 123 msec of the time series. scalp potential distribution produced largely by a single 
Classification decisions are made automatically by the 45 compact population of neurons, represented as a current 
computer, every 125 msec, using a nonlinear interpola- dipole, whose location is approximately known in the 
tion of neighboring decisions. Labeled segments (arti- subject's brain. A steepest-descent, non-linear, least- 
fact or no artifact) are used for each training class to squares fit between the measured potentials and the 
train the computer to distinguish artifacted segments, forward solution of the finite-difference problem is per- 
and an equal number for each test class. An interactive 50 formed by the computer softward. The unknown pa- 
graphics trial editing program is used by an operator to rameters include the ratios of conductivities of the tis- 
check the classification decisions of the computer pro- sues, as well as the dipole strength, and any parameters 
gram. Artifacts that are recoverable are filtered out of the dipole position and orientation left undetermined 
eg., non-saturating eye-blink are filtered out using least by anatomical knowledge gained from the MRI pic- 
squares noise cancellation with reference electrodes 55 tures. The absolute conductivities are not determined 
near the eyes; environmental MEG noise is filtered out from this measurement because the absolute strength of 
using least squares noise cancellation with reference the "equivalent dipole source" modeling this population 
sensors more than several centimeters away from the is not known* 

bead. There are a few brain sites which (i) produce a dis- 

G. Reduction of Volume Conduction Effects Using 60 *"* pic T* ^ ^T** °* ! W iCMU 

Spatial Deconvolution produce disUnct and recojpizable brain waves when 

F stimulated. For example, the brain waves which are 

1. Overview. An "image" of the brain wave electrical produced by the somatosensory evoked potential peak 
current normal to the surface of local cortical areas is at 20 msec produced by mild stimulation of the contra- 
achieved by using spatial deconvolution methods to 65 lateral median nerve at the wrist is a highly localized, 
reduce volume conduction and increase spatial resolu- discrete source. It is generated by a single tangential 
tion. Finite-difference methods are used to represent the dipole in the hand area of somatosensory area 36. This 
true geometry of the subject's brain cortex, cerebrospi- dipole is located 2.5-3.0 cm below the surface of the 
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scalp. Since the locus of the central sulcus is known without signal (low SNR). A noise process synthesized 

from the MRI pictures, a single parameter of position from the recorded data, as in the trial selection method, 

along the central sulcus is all that is required to determ- is used for the latter time series, 

ing the position of the activated dipole. In addition to 2. Steps in SNR Enhancement 

approximate locus and depth, published values of the 5 a. Create an artifact-free data set t from each subject's 

amplitude of the cortical potentials are also of use in stimulus-and-response-registered data. They will be 

constraining the solution. Only a single parameter of compared with a reference "baseline" data set, com- 

dipole orientation is required, since the dipole may be puted from the same data without stimulus-or-response- 

considered perpendicular to the sulcus and only its registration, in order to select trials with consistent 

degree of tilt into the sulcus is unknown. In a model 10 event-related signals. 

with 3 conductivity layers, there are only 5 parameters b. Compute average stimulus-registered (and re- 
being 2 conductivity ratios and 3 dipole parameters sponse-registered when appropriate) event-related time 
(including dipole strength). Using up to 256 scalp elec- series (ERTs) for each subject. Use the latencies of 
trades and an enhanced average evoked potential with chosen peaks in each person's averaged ERT time series 
a high SNR, an optimal or near-optimal solution for 15 to determine the centerpoints of analysis windows for 
these 5 parameters can be computed trial-selection analysis, see Gevins et al, Electrical Po- 
Another known compact source is produced by a tentials, etc., Science 213, pgs. 918-922 (1981) and 
stimulus at 1 degree to the right or left of the visual Neurocognitive Pattern, etc. Psychophysiology 22, pgs. 
fixation point This is mapped into a more-or-less radial 32-43 (1985), both incorporated by reference herein, 
(posterio-lateral) dipole in the contralateral occipital 20 Adjust the widow size to span the peak (usually 125 to 
pole. The subject is shown, as a stimulus, on the CRT 250 msec). Set other windows at stimulus and response 
monitor, a vertical constrast reversal sine wave grating, times. 

confined to a semi-^arcular area in either the right or the c. Apply a low-pass filter to the time series which 

left visual field. This type of stimulation produces a reduces power by 3 dB at 7 Hz and by 12 dB at 12 Hz; 

steady-state response produced by a single current di- 25 then decimate. Use 3-7 time series values as features, 

pole in the contralateral primary visual area. If two Partition the data of the two condtions (event-related 

parameters of orientation and one of dipole strength are data and noise) into three training and non-overlapping 

allowed, the two conductivity ratios again make five validation sets. 

parameters in the fit. d. For each channel, apply a layered-network pattern 

Recordings for these parameters are made at the end 30 recognition algorithm (a type of discriminant analysis), 

of the recording session after the active task data have see Gevins et al EEG Patterns etc., Ciiru NeurophysioL 

been collected. About 100 trials of each somatosensory 47, pages 693-703 (1979), incorporated by reference 

and visual stimulus are collected. herein, to generate equations which represent invariant 

H. SNR Enhancement „ Iu^Th^^ t cvent " r f Iate ? a " d "°™vent- 

35 related conditions in the current training data set at the 

1. Overview. Since brain waves are faint electrical or interval chosen. Test the equations with the validation 

magnetic signals, and the brain waves of interest which data to determine which electrodes differ significantly 

are related to a particular stimulus or response may be between conditions. 

obscured by other activity in the brain, it is important to e. Repeat step "d" (above) for each training and vali- 

improve the amount of stimulus— or response-related 40 dation data set and compute the mean validation set 

brain wave signal and decrease the non-stimulus— or accuracy. Set the level of significant classification well 

response-related "noise," i.e., improve the signal-to- above the level attained with the data randomly as- 

noise ratio (SNR). The SNR can be improved by simple signed to two pseudo-conditions, which are determined 

averaging of all trials or, optionally, by averaging only by a separate analysis. Draw diagrams showing which 

trials selected for a high SNR. This selection is done 45 electrodes contain significant task-related information 

with minimal assumptions about signal characteristic in each interval 

using a statistical pattern classifier to choose those trials f. Collect trials with task-related signals (i.e., trials 
which are discriminate from a synthesized noise pro- correctly classified in the validation set). Compute "en- 
cess with statistics very close to those of actual data hanced" averaged ERTs (and Wigner Tune-Frequency 
(FIG. 8), see Gevins, et al, 1986, EEG Clin. Neuro- 50 Distributions; see Morgan & Gevins (1986), cited infra), 
physiol, 66: 177-186, incorporated by reference herein. using those signal-bearing trials which attained signifi- 
Then, time and frequency bandpass filter to further cance in a majority of the channels. Graph ERTs, am- 
enhance the selected data are determined, optionally, plitude distributions over time intervals and Wigner 
with the aid of Wigner distributions see Morgan & Ge- Distributions, 
vins, 1986— IEEE Trans. Biomed Eng. BME-33(l) 55 

66-70, incorporated by reference herein. Filters are L Spa^ Synchronization ("Functional 

implemented with minimnm Heisenberg product (band- Interrelationship") Measures 

width x timewidth) designs. Between-channel covari- 1. Overview. Several methods measure "functional 

ances and other "functional interrelation" measures are interrelationships" of low-frequency, event-related 

computed on enhanced averages of the resulting filtered 60 brain signal time series as a first step in characterizing 

sets of (selected) single trial multivariate time series, the human brain's distributed processing network. This 

Filtering, as well as the trial selection, can be modified approach is based on the hypothesis that when areas of 

by feedback from classifier performance. This is called the brain are functionally related, there is a consistent 

"classifier-directed" filtering, see Gevins and Morgan, pattern of waveshape similarity between their brain 

1986, IEEE Tr. Biomed Eng., BAfE-33, (12), incorpo- 65 waves, i.e., predictability of the brain wave recorded at 

rated by reference herein. The chosen filter provides an one area at a particular time by that of other area(s) at 

optimality criterion of maximal discriminability be- earlier times to which it is functionally related. In previ- 

tween time series with signal (high SNR) and time series ous implementations, this amounted to measuring time 
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series correlation between pairs of channels on sets of this as a basis for transforming points is space, these 
single-trial data (Gevins et al., 1981, 1984, 1985, infra) points or vertices are combined to form edges and poly- 
Simpler and more robust computational procedures gons. Brightness values are coupled based on an ambi- 
have now been developed for measuring the magnitude ent lighting model and the polygon surface normals, 
and time delay of "functional interrelationships" from 5 Smooth surfaces are generated by interpolating the 
enhanced, filtered event-related, averaged timeseries. intensity values across each surface. Combining this 
These methods, which are collectively referred to as with a hidden surface removal algorithm, a realistic 
"event-related co variances" include: (1) optionally ap- image of any collection of polygons can be constructed, 
plying statistical pattern recognition methods for select- 2. Model head construction. The electrode sensors are 
ing trials with consistent event-related signals in order 10 marked in their normal positions on a mannequin head, 
to form the enhanced averages (see Section H, and Additional reference points are marked and numbered 
Gevins et al, 1986, cited infra); (2) detennination of over the remainder of the head. A list of polygons, in 
bandpass filter characteristics for the averages from computer memory, of the computer graphics system, is 
Wigner (time-frequency) distributions, see Morgan & then created where each polygon consists of a polygon 
Gevins 1986, cited infra, and/or by classifier-directed 15 ID number followed by an ordered list of reference 
filtering, (see Gevins & Morgan, 1986 cited infra); (3) points which make up the vertices of the polygon. Com- 
computation of crosscovariance functions between av- puter graphic calculations require that the polygons' 
eraged enhanced, filtered timeseries within brief surface be oriented by listing the vertices in a counter- 
(125-400 msec) intervals- centered on event-related clockwise direction as viewed from the outside of the 
waves; (4) determination of the significance of the co- 20 head. The resulting collection of vertex positions and 
variance by Monte Carlo statistical studies; and (5) the polygon lists is stored in a "display file" which can be 
use of 3-D perspective color graphics to display the read and displayed by the graphics display program, 
pattern of significant covariances, for example, on FIGS. 9 and 10. 

model heads or 3-D brain models reconstructed from Event-related covariances, and current distributions 

the MRI scan. 25 from single subjects or groups of subjects are displayed 

2. Steps in computing event-related convariances. on the model head using the following preferred con- 

a. Select digital filters and intervals for measurement ventions, see FIG. 10A. The thickness of a line is pro- 
of between-channel covariances by examining results of portional to the significance of the covariance. The 
step "f of SNR enhancement. color of a line indicates the time at which the two chan- 

b. Compute multilag crosscovariance functions be- 30 nels are maximally synchronized (yellow: 0-15 msec; 
tween all pair-wise channel combinations of the en- green: 16-31 msec; blue: 32-27 msec; red: 48-79 msec; 
hanced, filtered and decimated averages in each se- purple: 80 msec). The arrow points away from the chan- 
lected analysis window, see FIG. 9. Use the magnitude nel that is leading in time. The arrow is the same color 
of the m a x i m u m crosscovariance function and its lag as the line when the two convarying channels are posi- 
time as "functional interrelationship" features. 35 tively related. A gray arrow indicates that the covary- 

c. Determine the significance of the crosscovariance ing channels are negatively correlated. The current 
features. This requires an estimate of the standard devia- distribution of currents exiting and entering the scalp on 
tion of the "noise covariance, which is obtained as fol- a flame scale in which a maximal entering currents are 
lows: Random intervals in each single trial of the en- purple and maximal exiting currents are red. 

semble are averaged. The covariance analysis is per- 40 3. MRI reconstruction. Magnetic resonance images are 

formed on the filtered and decimated version of the used to provide the spatial locations of the surfaces of 

resulting averages, yielding a distribution of "noise" areas within the brain. Preferably the data provides ten 

covariances. or more sagittal, ten or more coronal and ten or more 

d. Graph the most significant covariances in each transaxial (horizontal) views of each subject. Each 
interval. The most significant covariances are the top 45 greyscale image is displayed on a color graphics corn- 
standard deviations) of the observed covariance distri- puter workstation and the contours of interest, i.e., the 
buttons, internal brain areas which are selected, are traced, by 

e. Test the ensemble of significant covariances for a the operator, using the mouse tracking device, see FIG. 
difference between experimental conditions. 3. The screen coordinates of the cursor are translated to 

For all significant covariances in either of the two 50 the corresponding image position, which is then used to 

conditions, compute the standard error and do a t-test compute the location within the magnet Inherent in 

between means. Measure similarity between multivari- each MRI image is a spatial error which is introduced in 

ate covariance maps with an estimate of the correlation the MRI process. By correlating the fiducial selectrode 

between them. Calculate these estimates using a distri- sensor markers from the MRIs, described above, with 

bution-independent "bootstrap" Monte Carlo proce- 55 the scale electrode sensor positions which are indepen- 

dure, which generates an ensemble of correlation values dently measured with an XYZ position transducer, 

from randomly selected choices of the repeated mea- described above, an error estimate is obtained. This 

sures. This will also yield a confidence interval for these estimate is used to provide a error correction for the 

estimates, internal brain contours described above. The corrected 

J. Thre^-Dimensional Perspective Graphics 60 COnt0 ™ m ^ t0 ° f t J? angles which r 

^ F cover the exterior surface outlined by the contours of 

1. Overview. 3-D graphics display the location and interest. Finally, the sagital coronal and transaxial re- 
appearance of objects in space as seen from an arbitrary constructions are combined to form a single computer 
viewpoint. The display is computed, by the computer graphic image, see FIG. 10B. 
software, as follows: A linear transformation in homo- 65 

geneous coordinates, contained in a single four-by-four K * lSou/re Localization 

matrix, is used to represent any combination of rotation, 1 . The Problem: Given an adequate model of the tissue 

translation, scaling and perspective transforms. Using of the head, a knowledge of the positions, multipole 
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character, orientations, and magnitudes of the sources minimize the sum of squared errors between predicted 

of the brain electrical potential and magnetic fields is and observed potentials. "Steepest descent" adjustment 

sufficient to determine the measured potentials and parameters can be calculated by setting to zero esti- 

fields at the scalp. Since the potentials may be consid- ma tes of the partial derivatives of the sum of squared 

ered quasistatic the spread of the potential is governed 5 errors with respect to each parameter. Ultimately, the 

by Poisson's equation: procedure should converge to a local minimum of error 

"v-firVA)-/ ^ ,e *' to a P 0 " 1 * *° r wn * c k anv incremental parametric 

adjustment results in an increase of squared error). Un- 
where <f> is the potential, o- is the conductivity tensor, m f °rtunately f ° r «"* constrained nonlinear optimization 
IMs the volume current source density. Considering 10 P^edures, there is no guarantee that any local mini- 
only isotropic models, in regions free of sources, this mum t fbund * a S* obal mmimum ' That *> there ma V 
reduces to Laplace's equation. other sets of parameters which better represent the 

data. This problem is especially bad for unconstrained 
y"(<rv»«o highly dimensioned problems, such as the problem of 

15 the identification of brain wave generating sites. 
Unfortunately, a knowledge of the potentials and fields b. Constrained Linear Least Squares. A brain model is 
at the scalp does not suffice to determine the sources. A posited with a restricted number of compartments 
given distribution of potential (field) at the scalp, i.e., (roughly 100) corresponding to major cortical areas and 
the brain generating sites, or any other surface including relevant subcortical systems, FIG. 11. The approximate 
the sources, may be produced by infinitely many differ- 20 location of compartments is determined from each sub- 
ent distributions of sources. In order to resolve this ject's MRI scan. Each compartment is represented by 3 
ambiguity, some additional information as to the char- orthogonally directed current sources. Four parameters 
acter, orientation, and/or number of sources must be mU st be determined for each source: I for position (the 
added to determine the sources of a given potential compartment number), and 3 for magnitude and orien- 
(field) distribution. Preferably the sources are modeled, 25 t ation (magnitude in each direction). This is a reduction 
in a computer software program, as one or a few equiv- from ^ 6 required for the unconstrained case. The 
alent current chpoles whose positions and orientations major advantage of this approach, however, is not the 
are unconstrained in a concentncsphere model of the 33% parameter reduction, but rather the ability to cal- 
tasue. However, while one or two equivalent current culate orie ntation from linear regression on subset vec- 
dipoles may be adequate to model the major source o a 30 ^ ^ a ^ of current densi ^ MEG distributions 

r?? 0 ^^ SUCh 03 t0 * ^ ^ Produced by each candidate source. The method unam- 

such models do not seem realistic for even simple goad- u:^^t„ j, 1 1 «. t r 

directed tasks. Furthermore, the nonspherical Wpology *f°?^ y ^ * e S^^?**?. «P lanatl ° n ° f 

of the subject's brain may result in considerable disto? *° ^ta, g,ven the hypothesized model order and he 

tion of the results 35 constraints of possible source regions. Even with the 

2. Overview of solutions: More realistic head and brain J^f 8 ^ fr< T . least-squares to linear 
models, as well as additional information, are required f 5 ^^.^ ^ <*™ d ^*™ still make 
to attempt multi-source localization. A number of steps the "J 1 *" difficult 1 ? e model mus } *f hl 6 h 
are applied to achieve these requirements: (1) use of enough » «PW«it the w eU; specifically, no 
MRI scans and finite-difference models to improve the 40 sources should actually represent multiple sources that 
head and brain model; (2) increased spatial sampling m sufficiently scattered to be in different functional 
using up to 256 scalp electrodes; (3) reduction of spatial areas ^ defmed bv compartmentalization). Table- 
smearing of scalp potential measures with improved dnven source modeling, employing "Divide-and-Con- 
Laplacian operators or fmite-difference spatial decon- subset-selection procedures, is used to make the 
volution; (4) use of other signal processing procedures 45 computations feasible, FIG. 12. 

ta enhance the SNR; (5) use of event-related covari- For each time P° mt m an interval with a stable be- 

ances of deblurred EEGs, and optionally MEG fields, tween-channel covariance pattern, subset-selection 

to constrain the errors in source modeling; (6) use of techniques are used to find the best N sets of up to 12 

several algorithmic approaches, including an exhaustive (depending on the number of EEG and/or MEG chan- 

search for a subset of physical sources from a larger 50 ne & used) hypothesized sources which best produce the 

candidate set; and (7) validation of the source models on observed EEG and/or MEG spatiotemporal patterns, 

independent data. e.g., many sets of 6 hypothesized sources, many sets of 

3. Use of several algorithmic approaches. The finite-dif- 5 hypothesized sources, etc. Subset selection is accom- 
ference (resistive network) approximation described plished by a successive approximation algorithm which 
above is extended to source modeling. The resistive 55 iteratively reduces the size of candidate compartments, 
network represents the scalp, skull, cerebrospinal fluid, For instance, an exhaustive search is conducted for the 
and the brain itself. Both cortical and subcortical best subset of up to six sources out of 50 compartments 
sources are postulated in least-squares minimization consisting of combined adjacent pairs from the original 
methods described below. Resulting brain currents corapartmentalization. The subset chosen can best pro- 
throughout the network are calculated. The calculated 60 duce the observed current density pattern, with the 
branch currents entering the skull predict the normal error weighted to emphasize those channels with larger 
current density due to each source. covariances with other channels. All subsets of up to 

a. Unconstrained nonlinear least squares. The historical size 6 (for the 50 compartment case) are computed for 
approach to source localization has been to posit the each of the decimated time points in a covariance inter- 
number of sources and then compute all 6 parameters (5 65 val. The choice of 6 is determined by the maximum 
for MEG) of each equivalent dipole using a least- number of subsets which can reasonably be regressed on 
squares adaptive minimization search. In this approach, with current computing resources (roughly 2 msec per 
source configurations are assumed and perturbed to regression on a 2-CPU Masscomp 5700 yields 2,300,000 
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possible subset evaluations in approximately 77 min- 
utes). Identification of 6 sources requires evaluation of 
24 parameters. One hundred twenty-eight to 256 elec- 
tric data points taken from high SNR averages is suffi- 
cient for this purpose. Higher numbers of sources may 
be evaluated by starting with a smaller number of larger 
compartments. In either case, succeeding steps repeat 
the subset selection to identify smaller volumes. 

As a methodological check, unconstrained nonlinear 
least-squares methods are used to locate the best single 
dipole to explain the variance of the observed potential 
distribution. This dipole should be located at the center 
of gravity of the linear least-squares solution. 

4. Compartmentalized brain model The purpose of the 
brain model is to define the loci of possible candidate 
sources for the source localization analysis. The follow- 
ing steps are required: 

a. Reconstruct a 3-dimensional computer graphic of 
the brain from MRI images. 



10 



15 



relatively stable for fairly long periods (50-100 msec or 
more), followed by brief chaotic transitions, followed 
by new relatively extended stable configurations. 
Therefore, there is imposed a maximum slope condition 
on the dipole solutions. 

The stability of each solution for multiple dipoles is 
examined over such intervals. Over-parameterized con- 
figurations are likely to change radially over neighbor- 
ing time points. A smoothness criterion is applied using 
a dynamic programming procedure to minimize error 
variance and between-sample variance over the inter- 
val. Under-parameterized configurations leave residuals 
(unexplained normal current densities and magnetic 
fields) which have significant topographic structure. A 
smoothness parameter weighting the significance of 
between-sample variance is perturbed to produce differ- 
ent solution model order trajectories over the chosen 
analysis interval. 
6. Cross-validation. The resulting time-varying models 



b. Scale the brain to standard coordinates based on 20 are tested on an independent data set (the original cn- 



the distance between the anterior and posterior commi- 
sures. 

a Form a uniform grid on the cortical surface with 
node points, taken as a subset from the fimte-difference 
resistive model, approximately 2.0-cm apart The sur- 25 
face of the grid conforms to the gross contour of the 
cortical surface but does not follow the foldings into the 
sulci, except for the longitudinal and sylvian fissures 
and central sulcus. 



semble of single trials is randomly split into 2 groups 
and subaverages formed). Squared error measures over 
the test set can be expected to increase for unrealisti- 
cally large model orders with parameters determined 
from the training set. 

The principal criteria for assessment of a particular 
multiple source solution are: (1) it should make sense. A 
source pattern should be consistent with prior 
neuroanatomies and neurophysiologies knowledge, 



d. At each node point, determine the 3-dimensional 30 i.e., the location of sensory and motor sources should be 



coordinates of the point halfway from the surface to the 
deepest sulcus, along the normal projection below the 
surface (approximately 1.5 cm). This point is the loca- 
tion of a candidate equivalent dipole source which 
could be modeled by injecting current at this point in 35 
the resistive network. This equivalent dipole represents 
the resultant vector of all the dipole generators in a 
volume aligned normal to the surface of the grid, with 
approximately 1.5-cm radius and 1.5-cm depth. Because 
of the likelihood of cortical folding within this volume, 40 
the equivalent dipole may have any orientation. There- 
fore 4 parameters are required to specify each equiva- 
lent dipole: 1 to identify the node, and 1 for current 
strength in each of the 3 coordinate directions. Roughly 
90 such candidate equivalent dipole sources are re* 45 
quired to adequately sample both cortical hemispheres 
with 3-cm node separation. 

e. In addition to the candidate cortical sources, select 
a set of candidate subcortical source structures. Most 



correct; (2) it should produce an increase in signal-to- 
noise ratio, where the signal is measured by the signifi- 
cance of hypothesis tests of the between-channel co- 
variance and the noise is measured from a noise distribu- 
tion of co variance; (3) lower-order solutions (such as a 
single dipole) should be located at the center of gravity 
of clusters of the multiple source solution; (4) the source 
model should be a good representation (i.e.*, have low 
residual variance) for an independent set of test data. 

Numerical simulations are conducted to determine an 
error criterion for the candidate selection process. All 
smoothness and cross-validation tests are conducted on 
a large number of candidate source combinations that 
fall within the error criterion of the minimum error 
subset This is a safeguard against gross errors caused by 
incorrect choices in the initial steps of the successive 
approximation search. The simulations also provide a 
measure of the spatial resolving power of the instrument 
for each particular brain topology. In particular, statis- 



subcortical locations are excluded a priori because on 50 tics evaluated in a Monte Carlo simulation of position 



theoretical grounds they could not produce a discern- 
ible scalp potential or magnetic field. Prime candidates 
are telencephalic or diencephalic areas with structures 
which could produce an open field of potential. These 
include hippocampus, lateral geniculate bodies, ventre- 55 
posterior and ventrolateral thalamic nuclei. 

f. Determine the 3-dimensional locations of the sub- 
cortical candidate sources from the MRIs. Most of the 
candidate structures are modeled by one equivalent 
dipole. The hippocampus, because of its geometry, re- 60 
quires three separate dipoles to represent its various 
sub-regions. Thus, in all, there are a total of roughly 102 
candidate sources. 

5. Determining the Number of Sources. As with all 
regression problems, the correct parameterization is 65 
critical to a model that is generalizable and sufficiently 
represents the sources of variance in the data. The con- 
figuration of the scalp potential Meld tends to remain 



errors provide a confidence interval for source localiza- 
tion results. 
What is claimed is: 

1. A method to display the location and inter-area 
temporal relationships of a human subject's brain neural 
activity, comprising the steps of: 

0) determining and recording the three-dimensional 
location of selected portions of the subject's head 
and internal portions of the subject's brain and 
recording said locations using a three-dimensional 
anatomical scan instrument to provide anatomical 
location data; 

fu) positioning a plurality of at least 32 sensors exter- 
nal to the subject's scalp and proximate to the sub- 
ject's scalp to detect analog brain neural activity 
detectable proximate to the surface of the scalp; 

(iii) determining the three-dimensional positions of 
the sensors, converting the said sensor position 
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determinations into digital data, and storing the 
said sensor position digital data in computer stor- 
age means; 

(iv) providing a set of predetermined stimuli to the 
subject; 5 

(v) detecting the subject's analog brain signals at the 
sensors during delivery of said stimuli and amplify- 
ing said analog signals; converting the analog sig- 
nals into digital data and storing said digital data in 
computer storage means; 10 

(vi) statistically analyzing (a) the said anatomical 
location data, (b) the said sensor position data, and 
(c) said stored brain signal digital data, to provide a 
set of statistical indications which represent gener- 
ators of brain neural activity at sites within the IS 
brain and their temporal interrelationship; and 

(vii) displaying said statistical indications on a three- 
dimensional perspective display of at least part of a 
human brain, the brain portions of said display 
corresponding in location to the brain portion loca- 20 
tions provided by the scan instrument. 

2. The method as in claim 1 wherein the location of 
the selected portions of the brain is obtained by scan- 
ning the brain to provide magnetic resonance images 
(MRI). 25 

3. A method as in claim 1 wherein the brain wave 
signals are improved in their signal-noise ratio to reduce 
contaminants and reduce brain activity not related to 
said stimulus by comparing trial sets of said brain wave 
digital data to statistically determined norms of uncon- 30 
taminated and non-stimulus related brain wave digital 
data in a pattern recognition classification to eliminate 
those data which are contaminated or which do not 
have strong stimulus-related brain wave signals. 

4. A method as in claim 1 wherein the sensors are 35 
electrodes positioned on the internal face of a remov- 
able hat 

5. A method as in claim 1 wherein the sensors are 
electrodes of an electroencephalograph (EEG) system. 

6. A method as in claim 1 wherein the sensors are 40 
detectors of a magnetoencephalograph (MEG) system. 

7. A method as in claim 1 wherein the stimuli are 
automatically presented under computer program con- 
trol. 

8. A method as in claim 1 wherein the stimuli include 45 
a set of motor tasks to be performed by the subject 

9. A method as in claim 8 wherein the tasks include a 
task in which the subject must respond with left or right 
hands to a visual cue. 

10. A method as in claim 8 wherein the subject must 50 
respond with one hand to sound and visual cues. 

11. A method as in claim 1 wherein said display in- 
cludes color coded lines and areas in which color of 
lines indicates time delays and color or areas indicates 
current flow through the scalp. 55 

12. A method as in claim 1 wherein the display is a 
model based on the subject's brain and head. 

13. A method as in claim 1 wherein the electrical 
brain wave digital data is processed to reduce distortion 
due to passage from the originating sites to the sensors, 60 
said processing including the use of a spatial deconvolu- 
tion algorithm. 

14. A method as in claim 1 wherein the electrical 
brain digital data is processed to extract source network 
properties using compartmentalization, subset selection, 65 
and linear least squares methods. 

13. A method as in claim 1 wherein the subject per- 
forms a predetermined task consisting of a series of trials 



substantially involving the voluntary movement of a 
portion of the subject's body in response to the stimulus 
and the detection of the subject's brain waves continues 
during each trial. 

16. A method as in claim 1 wherein the display is of 
the spatial and temporal brain wave properties of stimu- 
lus-or response-related signals, including preparation, 
stimulus encoding, feature-extraction, decision, re- 
sponse preparation, response execution, feedback- 
updating, and other cognitive processes. 

17. A method as in claim 1 in which the display is a 
model of the subject's brain and head, the scan is a 
magnetic resonance image (MRI) and the model is 
based on the said subject's MRI. 

18. A system to display the location and inter-area 
temporal relationships of a human subject's brain neural 
activity, comprising: 

(i) a plurality of at least 32 sensors adapted to be 
positioned proximate the subject's scalp to detect 
analog brain neural activity detectable proximate 
to the surface of the scalp; 

GO means for determining the three-dimensional posi- 
tions of the sensors, converting the said position 
determinations into digital data, and storing the 
said position digital data in computer storage 
means; 

(iii) means connected to said sensors for detecting and 
amplifying the subject's analog brain signals; con- 
verting the analog signals into brain signal digital 
data and storing said brain signal digital data in 
computer storage; 

(iv) means for statistically analyzing the stored brain 
signal digital data to provide a set of statistical 
indications of brain activity representing temporal 
interrelationships between sensor positions; 

(v) a three-dimensional perspective display means 
having head and brain portions for displaying said 
statistical indications as lines between sensor posi- 
tions; where the said head and brain portions on 
said display means correspond in location to the 
sensor positions. 

19. A system as in claim 18 and further including 
brain scan means to determine and record the three-di- 
mensional location of internal portions of the subject's 
brain, wherein the brain scan means is a magnetic reso- 
nance image (MRI) system. 

20. A system as in claim 18 and including a pattern 
recognition classification means to eliminate those data 
set trials which do not have strong stimulus-related 
brain wave signals. 

21. A system as in claim 18 and including a removable 
hat on which are mounted at least 32 of said sensors. 

22. A system as in claim 18 and including a computer 
program control means to automatically present said 
stimuli. 

23. A system as in claim 18 wherein the display is a 
model based on the subject's brain. 

24. A system as in claim 18 wherein the sensors are 
electrodes of an electroencephalograph (EEG) system. 

25. A system as in claim 18 wherein the sensors are 
detectors of a magnetoencephalograph (MEG) system. 

26. A method as in claim 18 wherein the electrical 
brain wave digital data is processed to extract source 
network properites using compartmentalization, subset 
selection, and linear least squares methods. 

27. A system to display the location and inter-area 
temporal relationships of a human subject's brain neural 
activity, comprising: 



04/22/2004, EAST Version: 1.4.1 



4,736,751 

21 22 

(0 brain scan means to determine and record the statistical indications as lines between sensor posi- 

three-dmiensional location of selected internal por- tions; wherein the said head and brain portions on 

turns of the subject's brain and head by viewing said display means correspond in location to the 

said selected brain portions in a three-dimensional sensor positions. 

scan; 5 28. A system as in claim 27 wherein the sensors are 

00 a plurahty of at least 32 sensors adapted to be electrodes of an electroencephalograph (EEG) system, 

posiuoned proamate the subjecrt scalp to detect 29 . A ^ ^inchum* he sen J rs ^ 

analog brain neural activity detectable proximate a<*+~** ~r t » , Z X*WXv 

to the surface of the scalp; ? 0f ? ^etoencephalograph (MEG) system. 

Oii) means for determining the three^iunensional 10 * t^-.T^V ^ deCtnCal 

positions of the sensors, converting the said posi- bnUn T d,gltal da ! a 13 P rocessed t0 extract source 

tion determinations into digital data, and storing n * tW P^P^ usln S compartmentalization, subset 

the said position digital data in computer storage ™^ and least s< » uares methods, 

means; 31. A system as in claim 27 wherein the brain scan 

Civ) means for providing a set of predetermined stim- 15 means is a ^e^tic resonance image (MRI) system, 

uli to the subject; ^2. A system as in claim 27 and including a pattern 

(v) means connected to said sensors for detecting and recognition classification means to eliminate those data 
amplifying the subject's analog brain signals which set trials which do not have strong stimulus-related 
are responsive to said stimuli converting the ana- ^raui wave signals. 

log signals into brain signal digital data and storing 20 ^ A system as in claim 27 wherein said sensors are 

said brain signal digital data in computer storage; mounted on a removable hat. 

(vi) means for statistically analyzing the stored brain 34. A system as in claim 27 and including a computer 
signal digital data to provide a set of statistical program control means to automatically present said 
indications of brain activity representing temporal stimuli. 

interrelationships between sensor positions; and 25 35. A system as in claim 27 wherein the display is a 

(vii) a three-dimensional perspective display means model based on the subject's brain, 
having head and brain portions for displaying said * * * * * 
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[57] ABSTRACT 

A transcutaneous medical electrode is provided for 
coupling electrical energy into nerves and/or muscles 
for the stimulation thereof. The electrode uses an array 
of conductive ink patterns in combination with a con- 
ductive adhesive to provide power coupling in which 
the current density across the electrode can be prese- 
lected by ink design for use on specific curvaceous areas 
of a body to transcutaneous^ stimulate muscle and/or 
nerves. 

25 Claims, 1 Drawing Sheet 
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